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ABSTRACT 

This report is designed to help policymakers, parents, 
reporters, and foundation officials better understand what has been learned 
about voucher programs and why some key questions remain unsettled. It is 
based on an analysis of voucher programs in Milwaukee, Cleveland, and Florida 
State. Four basic questions are addressed: (1) What do we know from current 

studies of publicly funded voucher programs? (2) What more could we learn 
with additional research on these programs? (3) What key questions cannot be 
answered because of how existing program are structured? and (4) What can 
policymakers, researchers, and funders of research do to expand knowledge of 
vouchers? Key findings show that there is inconclusive evidence on the 
effectiveness of vouchers for improving education, that studies on vouchers 
have varying findings about student achievement, that voucher examples from 
other countries have not been fully examined, and that some questions will be 
difficult to answer no matter which research method is used. Recommendations 
include up-front evaluation requirements when authorizing a voucher program, 
performing long-term high-quality evaluation, and performing multiple kinds 
of studies. The report concludes with 19 references and an appendix 
containing major research questions that should be addressed by evaluations 
of any voucher program. (RT) 
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About the Center on Education Policy 

The Center on Education Policy is the national independent advocate for public 
education and for more effective public schools. The Center works to help Americans 
better understand the role of public education in a democracy and the need to improve 
the academic quality of public schools. We do not represent any special interests. Instead, 
we help citizens make sense of the conflicting opinions and perceptions about public 
education and create conditions that will lead to better public schools. 

Working at the national, state, and local levels, the Center achieves its mission by 
producing publications, writing articles, convening meetings, making presentations, and, 
upon request, providing expert advice. The Center also works jointly with many other 
education, business, government, and civic organizations. 

Based in Washington, D.C., and founded in January 1995, the Center receives nearly 
all of its funding from charitable foundations. To learn more about our work, please 
visit our web site at www.ctredpol.org. 
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I NTRODUCTION 



Why People Want Objective [Information about Vouchers 

School vouchers, sometimes called tuition scholarships, are one of the most polarizing issues in 
education today. Vouchers are payments that parents can use to send their child to a private or 
religious school. Most current voucher programs are targeted at low-income families. (Some states 
and school districts also have public school choice programs, which allow children to attend a 
different public school than their neighborhood school, but these arrangements are not usually 
called vouchers.) Although many voucher or scholarship programs are financed through private 
sources, the most hotly debated forms of vouchers are funded with public tax dollars and subsidize 
tuition at private and religious schools. 

Current voucher programs have been the subject of several studies — which sometimes reach 
conflicting conclusions. A newspaper will one day feature a study saying vouchers are successful 
in improving student performance or achieving other goals, then the next day the same paper will 
report on a different study suggesting vouchers are a failure. In this highly charged climate, it is 
difficult for parents and others to sort through conflicting evidence and know what to believe. The 
issue is further clouded by people on either side who selectively use research as political ammunition 
and scholars who quarrel about each other’s research methods. The average citizen doesn’t 
understand why the experts can’t give clear or consistent answers — or in some cases, any answer 
at all — to pressing questions about the effects of vouchers. 

This Report Addresses Four Main Questions 

The Center on Education Policy has produced this report to help policymakers, parents, reporters, 
and foundation officials — and anyone else who isn’t a researcher — better understand what has 
been learned from various studies of voucher programs and why some key questions remain 
unsettled. We also recommend steps that can be taken to answer those questions, through additional 
studies that meet high standards of objectivity and professionalism. Our ultimate goals are to 
encourage respectful discussion about voucher studies among people with different views and to 
instill greater public confidence in the research process. 

This report is based on an analysis of voucher programs in three sites: the publicly funded voucher 
programs currently operating in the cities of Milwaukee and Cleveland and the new voucher program 
in the state of Florida. We also have considered the array of programs in other countries that provide 
subsidies, vouchers, or general aid to private and religious schools. 
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The report addresses four basic questions: 

1. What do we know from current studies of publicly funded voucher programs? 

2. What more could we learn with additional research on these programs? 

3. What key questions cannot be answered because of how existing programs are structured? 

4. What can policymakers, researchers, and funders of research do to expand knowledge 
of vouchers? 

This report focuses on studies of existing, publicly funded voucher programs. The advisory panel 
that assisted us did not reach consensus about whether to examine studies of privately funded 
voucher programs, so they are not addressed in this report. In addition, vouchers supported through 
public taxes are more controversial than privately funded programs and are on the policy agendas 
in several states and communities. A fresh, straightforward look at the research on publicly funded 
programs could be a welcome contribution to national, state, and local public policy debates. 
Although studies of privately funded voucher programs have produced interesting and useful 
findings, these programs operate outside the public policy arena, do not raise issues of appropriate 
use of tax dollars, and do not require direct involvement of state and local policymakers. 

The Centers work was greatly informed by thoughtful advice from a diverse panel of experts, as 
explained later in this report. But this report represents the views of the Center on Education Policy 
alone and is not intended to speak for any other organization. 

Organization ofThis Report 

The sections of the report that follow: 

— summarize key findings and recommendations; 

— describe the process used to produce this analysis; 

— briefly describe the voucher programs and studies analyzed; 

— explain in more detail what we have learned from current voucher studies; 

— discuss what more could be learned about vouchers with further study; and 

— discuss which questions cannot be resolved without changes in policies or programs. 
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SUMMARY OF KEY FINDINGS AND 
RECOMMENDATIONS 



Key Findings about Voucher Studies 

What do we and don’t we know about publicly funded voucher programs? The Center has reached 
a few key findings: 

O Inconclusive evidence* 

The jury is still out on whether vouchers are an effective policy for improving 
education and what effects they have on student achievement, on other student 
outcomes, and in other areas. No single study — or even several current studies 
— should be viewed as definitively answering these questions. 

Q Varying findings about achievement. 

Studies of current, publicly funded voucher programs have focused mainly on 
the academic achievement of students who use vouchers, compared to their 
counterparts who remain in public school. These studies have reached different 
conclusions, with some finding achievement gains for voucher students and others 
finding no improvement. Additional high-quality studies could help to reconcile 
these contradictions and determine the effects of voucher programs on students 
who remain in public schools, including those who are offered vouchers but do 
not use them. 

O Impacts beyond achievement. 

There is a strong need for more high-quality studies that look beyond student 
achievement to examine such issues as which other effects vouchers have on 
students, how well programs are being implemented, whether schools are 
complying with student selection and other requirements, why some families do 
not apply, and why some students leave voucher programs. 

O Effects on schools. 

Major questions that have not been answered but could be pertain to the impact of 
voucher programs on public, private, and religious schools, such as how vouchers 
affect curriculum, instruction, and operations of participating schools and whether 
voucher programs lead to positive or negative changes in public and private 
educational systems. 
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O Other countries. 

Many other nations have programs that provide choice, vouchers, subsidies, or aid 
to private and religious schools. Studies of these programs have not been fully mined 
for insights relevant to the U.S. A comprehensive, balanced, and rigorous review 
of international studies would be a relatively low-cost way to expand our knowledge. 

0 Challenges of research. 

The challenges involved in designing an objective voucher study are substantial 
enough that some questions will be difficult to answer no matter which method is 
used. This explains why reasonable people sometimes disagree and why studies 
usually contain qualifiers or caveats. 



Recommendations for Policymakers 

What actions can state and local legislators and other policymakers take to promote high-quality, 
objective information about publicly funded voucher programs? What kinds of information will 
help them determine whether voucher programs are working as intended? The Center recommends 
that policymakers consider the following points if their state or community has a voucher program 
or is weighing a voucher proposal: 

0 Up-front evaluation requirements. 

Legislation authorizing a voucher program should include an evaluation component 
from the very beginning. The legislation should require public schools and 
participating private and religious schools to collect and make available consistent 
student achievement data and other key program information. Policymakers should 
also consider how necessary evaluations will be funded. 

O Questions to study. 

In developing evaluation requirements for a voucher program, policymakers should 
consider the comprehensive list of questions in the appendix, which was developed 
by a group of voucher proponents, opponents, and others. In general, evaluations 
should address the program’s impact on participating and non-participating 
students, parents and families, and public, private, and religious schools. They 
should also look at program implementation and cost issues. 

O Need for longer-term evaluation. 

Evaluation requirements should not be eliminated too early, just because a program 
has evolved beyond the “experimental” stage. As programs mature and change, 
new questions arise and behaviors change. Because some effects take time to show 
up, evaluations should track students, families, and schools over a period of years. 
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Recommendations for Researchers and Funders of Research 

What can researchers and groups that fund research do to expand knowledge of vouchers, improve 
the quality of voucher research, and “lower the volume” of voucher debates? Most obviously, funding 
organizations can provide sufficient resources for needed research. In addition, the Center 
recommends the following actions for researchers, foundations, nonprofit organizations, and other 
groups that conduct or support research: 

O Representing different views. 

Certain processes could produce more fruitful and balanced discussions of voucher 
studies among people with different positions. For example, researchers and funders 
could include people with differing viewpoints on study advisoiy boards and among 
early reviewers. Conferences could encourage people with different perspectives 
to present and discuss draft papers before publication. 

O Questions to study. 

Voucher studies should look more deeply at a variety of issues that go beyond 
student achievement. In designing studies, researchers and funders should consider 
the comprehensive list of questions in the appendix. If the evaluation cannot 
address all of the key questions due to budget constraints or other reasons, then 
evaluators should make explicit choices about what to tackle, after taking into 
account the relative importance of various issues, the need for serious study of 
neglected issues, the particular strengths of their evaluation team, and the topics 
that could best be addressed by somebody else. 

O Multiple kinds of studies. 

There is a need for multiple kinds of voucher studies that tackle questions from 
different vantage points. For example, statistical analyses should be enriched by 
approaches such as classroom observations, focus groups, and surveys of educators 
and policymakers. 

O High-quality evaluations. 

Voucher studies should be designed, carried out, reviewed, and reported according 
to professional ground rules for objective and rigorous evaluation. Studies should 
clearly describe the methods used, so they will be transparent to other researchers. 

O School cooperation. 

Good voucher research requires cooperation from the public, private, and religious 
school sectors. Researchers should seek involvement of public, private, and 
religious school officials early in the evaluation process and should take steps to 
convince educators from both sectors that their concerns will be handled fairly 
and objectively. 
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PROCESS FOR THIS ANALYSIS 
AND ROLE OF THE EXPERT PANEL 



In July 1999, the Center on Education Policy convened an expert panel, initially to discuss the 
feasibility of mounting a large-scale evaluation of voucher programs. Participants included 
researchers who had studied voucher programs or related issues, people with expertise in education 
policy, representatives of public, private, and religious school groups, and officials from foundations 
that fund education studies. Among the participants were nationally known proponents and 
opponents of vouchers. All the participants in these meetings are listed on the Center’s web site at 
www.ctredpol.org. 

The panel could not reach consensus about a large-scale evaluation, but it did identify other tasks 
it could do to advance the state of voucher research. The panel met twice more, in October 1999 
and January 2000. Smaller subgroups met three other times. 

As a first task, the panel discussed the kinds of questions that should be addressed by all objective 
evaluations of a voucher program. A subgroup refined and organized the questions into a list. The 
larger group reached consensus on this list, which appears in the appendix. Agreement on this list 
among such a diverse group is a noteworthy accomplishment in itself. 

As a second task, the panel analyzed the major studies of publicly funded voucher programs to 
determine what is known and not known about the effects of vouchers and to identify productive 
areas for future research. Much of the analysis was done by a smaller subgroup, further divided 
into four subcommittees. Three subcommittees analyzed the three U.S. sites (Milwaukee, Cleveland, 
and Florida). A fourth subcommittee looked at programs in other countries. The subcommittee 
reports, along with a complete list of subgroup members, can also be found on the Center’s web 
site (www.ctredpol.org). 

The work of the panel was enormously valuable in the development of this report. But the Center 
did not ask panel members to agree to any formal recommendations or endorse this report. The 
participation of individuals on the panel should not be interpreted to mean either agreement or 
disagreement with our findings or recommendations. 
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DESCRIPTIONS OF VOUCHER PROGRAMS 
AND THEIR MAJOR STUDIES 



Three publicly funded voucher programs are currently operating in the U.S., in Milwaukee, 
Cleveland, and Florida. Table 1 compares their key features. Several other nations also operate 
programs that offer choice, subsidies, or vouchers for private and religious schools. 



TABLE 1 

Key Features of Three Publicly Funded Voucher Programs (May 2000) 




MILWAUKEE 


CLEVELAND 


FLORIDA 


YEAR 

ENACTED 


1990 


1995 


1999 


FUNDER 


State of Wisconsin 


State of Ohio 


State of Florida 


NUMBER OF 
STUDENTS 
USING 
VOUCHERS 


About 7,900 in preK- 
1 2; could rise to 1 5% 
of public enrollment 


About 3,600 in K-5 


1 19 in K-12, could go 
higher if more schools 
deemed failing 


ELIGIBLE 

CHILDREN 


Family income up to 
1 75% of poverty line 


Priority to families 
with incomes up to 
200% of poverty line 


Students in attendance 
areas of public schools 
deemed failing 


VOUCHER 

SELECTION 


Schools must accept 
eligible students; 
lottery (with 
conditions) if 
oversubscribed 


Lottery with 
conditions 


Schools must accept 
eligible students; 
lottery (with 
conditions) if 
oversubscribed 


VOUCHER 

AMOUNT 


Up to $5, 1 06 in 
school year 99-00 


Up to $2,250 in 
school year 99-00 


Up to about $4,000 in 
school year 99-00 


TYPES OF 
SCHOOLS 


Private and religious 
schools 


Private and religious 
schools 


Private and religious 
schools (or public 
schools with 
acceptable ratings) 


COURT 

DECISIONS 


State Supreme Court 
ruled constitutional in 
1998; U.S. Supreme 
Court let state ruling 
stand 


Federal judge ruled 
unconstitutional in 
1999; under appeal 


Circuit court judge 
ruled unconstitutional 
in 2000; under appeal 
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Milwaukee Parental Choice Program 

Program description. In 1990 the Wisconsin legislature enacted the first major publicly 
funded voucher program in the nation, the Milwaukee Parental Choice Program. Under the current 
program rules, students in grades K through 12 who live in Milwaukee may apply for vouchers of 
up to $5,106, if their family income does not exceed 175 percent of the federal poverty level. 
Vouchers may be used at private and religious schools in Milwaukee. The original legislation 
restricted the program to non-religious private schools, but amendments in 1995 widened it to 
encompass religious schools. This expansion, however, was challenged in court, and religious 
schools did not actually begin to participate until 1998, when the Wisconsin Supreme Court ruled 
their involvement to be constitutional. 

As of January 2000, there were 93 private and religious schools participating. The authorizing 
law requires voucher schools to accept all eligible students who apply, unless the schools are 
oversubscribed; then they must select students randomly, with a few exceptions. In school year 
1999-2000, about 7,900 students, or roughly 8 percent of the Milwaukee public school enrollment, 
actually used a voucher — well below the 15 percent cap on participation in the authorizing law. 

Studies of Milwaukee. The Milwaukee program has been the most heavily studied of the U.S. 
sites. The original authorizing law called for yearly evaluations, which were carried out by John 
Witte and colleagues from the University of Wisconsin. During the program’s first five years of 
operation, the Witte group collected and analyzed student achievement data and studied other 
aspects of the program (Witte, Sterr, and Thorn, 1995; Witte and Thorn, 1994). Two other studies 
independently analyzed Wittes raw data: a study by Jay Greene of the University of Houston and 
Paul Peterson and Jiangtao Du of Harvard University (Greene, Peterson, and Du, 1997); and one 
by Cecilia Rouse of Princeton University (Rouse, 1998). But Milwaukee’s value as a research site 
was severely compromised in 1995, when the legislature did away with requirements for schools 
involved with the program to collect or report achievement data. The only evaluation currently 
mandated is a five-year review by the Wisconsin Legislative Audit Bureau (Wisconsin Legislative 
Audit Bureau, 2000). 



Cleveland Scholarship and Tutoring Grant Program 

Program description. Enacted by the Ohio legislature in 1995, the Cleveland Scholarship and 
Tutoring Grant Program provides tuition vouchers to eligible students in grades K through 5 to 
enable them to attend private or religious schools. (Eligible parents may choose instead to keep 
their children in public school and apply for “tutoring grants” for extra academic help.) Priority 
for vouchers goes first to families with incomes at or below the poverty level, and next to families 
with incomes up to twice the poverty level. If any scholarships remain, families with higher incomes 
may apply. The distribution of voucher recipients must reflect the proportion of African American 
students in the Cleveland public schools, and no more than 25 percent of the vouchers may go to 
children who were previously enrolled in private schools before becoming eligible. After these 
conditions are met, vouchers are to be awarded by lottery. The maximum voucher is the lesser of 
$2,250 or 90% of actual tuition at the chosen school. 
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In 1998-99, about 3,600 students participated — less than 7 percent of the total Cleveland public 
school enrollment. In December 1999, a federal judge declared the Cleveland program to be 
unconstitutional, based on church-state grounds, and suspended new applications. Participating 
students may continue while the decision is being appealed. 

Studies of Cleveland. The authorizing legislation mandated an independent evaluation, which 
is being carried out by Kim Metcalf and his team from Indiana University (Metcalf et al., 1998; 
Metcalf et al., 1997). This study is slated to continue for several more years, with plans to follow a 
first-grade cohort of students at least through sixth grade. Researchers Jay Greene, Paul Peterson, 
and William Howell have done their own analyses of the Metcalf data (Peterson, Greene, and Howell, 
1998), and have also studied student achievement and parent satisfaction in two Cleveland voucher 
schools (Greene, Howell, and Peterson, 1997). Other reviews have looked at program management, 
finance, and implementation (KPMG, 1999; Petro, 1999a and 1999b). 



Florida Opportunity Scholarship Program 

Program description. The Florida Opportunity Scholarship Program, the first statewide 
voucher program, is part of a broader education reform package enacted in 1999. Under this 
new approach, the state education department rates all Florida public schools on a scale of “A” 
to “F”. Highly rated or improving schools are rewarded with extra funding and deregulation, while 
schools rated F are designated as “failing.” Students in grades K through 12 who attend schools 
that receive an F grade for two out of four years become eligible for vouchers. These vouchers may 
be used to attend a religious or private school, or a public school rated C or higher. 

The voucher component was first implemented in school year 1999-2000. One hundred nineteen 
students from two failing schools, or roughly 10 percent of the two schools’ enrollments, accepted 
vouchers. Of the 119 voucher users, 52 went to private schools and the rest to other public schools. 
In the next two years, some 20 percent of Florida students could become eligible for vouchers, 
based on public school ratings so far. 

The voucher amount ranges from about $3,400 to $4,100 but cannot exceed the actual cost of tuition 
at the receiving private school. The legislation requires schools that are oversubscribed to select 
voucher students on a random and religiously neutral basis, except that preference may be given 
to siblings of voucher students already attending. Voucher students must take annual state 
achievement tests, although private and religious schools attended by voucher students are not 
required to administer state tests (in which case students must go to another testing center). In 
March 2000, a circuit court judge declared the Florida voucher program unconstitutional; the 
decision is being appealed to a higher court. 

Studies of Florida. A joint team of researchers from Princeton University, the University of 
Florida, and the Urban Institute is conducting a comprehensive independent evaluation of the 
Florida voucher program. This study is just getting underway. 
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International Programs 

Program description. Many other nations have choice, voucher, subsidy, or aid programs that 
involve private and religious schools. Australia subsidizes private schools based on enrollment 
and need. The Netherlands has long funded private schools on an equal basis with public schools. 
Several other European nations — Belgium, Denmark, and France, to name just a few — also 
provide various forms of aid to private schools. Chile, Columbia, and certain Canadian provinces 
have implemented programs similar to vouchers. 

Although only a few of these international programs are similar to vouchers as we define them in 
the U.S., many countries provide subsidies or general aid to private and religious schools. These 
programs often operate within educational systems and cultural contexts rather different from ours. 
In some countries, for example, religious schools are part of the government education system and 
must follow national curriculum and other requirements. Yet even with these differences, the 
experiences of other nations could shed light on aspects of vouchers now being debated in the U.S. 

Studies of the international context. Private school subsidy, voucher, and choice programs 
in other countries have been closely studied, but the findings are not widely known in the U.S. 
Numerous studies address pivotal questions for which we have little evidence in the U.S. such as 
how choice affects social stratification over time; what impact government regulations have on private 
and religious schools; how government subsidies affect the start up of new private schools and 
whether new schools differ from established ones; what happens to public schools that lose a critical 
mass of students; and how private school subsidies affect private school costs and spending for 
public education. 



General Observations about Program Structure and Rules 

Some common observations can be made about the publicly funded voucher programs in Milwaukee, 
Cleveland, Florida, and other countries: 

O The rules established up front make an enormous difference in how the voucher 
program operates, how people react to it, and what can be learned from it. 

When policymakers enact a voucher program, they face a variety of decisions, such 
as who will be eligible, how much vouchers will be worth, how many participants 
the program will support, which schools will participate, how schools will select 
voucher students, which requirements will be placed on private schools, and how 
programs will be evaluated. The decisions they make really matter. So do the 
decisions they don’t make, either intentionally or through omission, such as 
refraining from regulating private schools. 

O State and local context affects how a voucher program operates and whether 
findings about one program can be generalized to another. 

Voucher programs are shaped by such factors as urban or rural location, culture 
and demographics of the community, historical approaches to local governance, 
and local traditions of religious education. Another important contextual factor is 
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the availability and cost of private and religious schools in the community. If the 
contexts are too dissimilar, research findings about one voucher program may have 
limited application in other sites. 

O Legal challenges and court rulings have created uncertainty in voucher programs. 

Ongoing litigation and mixed rulings by state and federal courts about the 
constitutionality of vouchers have created instability in program operations and 
uncertainty among parents, policymakers, and school officials. It is unclear how 
this legal situation has affected participation, school actions, or policy decisions, 
but the uncertainty seems unlikely to go away for some time. 

O Publicly funded voucher programs in the U.S. have been subject to surprisingly 
little research compared with the attention they have attracted. 

Much has been made about the relatively few studies of publicly funded programs. 
A lack of cooperation among people on different sides of the issue has probably 
inhibited researchers from undertaking other studies. There is a need for additional 
high-quality research on current programs. 
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WHAT WE KNOW FROM STUDIES 
OF PUBLICLY FUNDED VOUCHERS 



What do recent studies tell us about publicly funded voucher programs? With the help of an expert 
panel, the Center has analyzed major studies of Milwaukee and Cleveland and identified several 
tentative themes and patterns, as well as challenges that complicate the design of voucher research. 



Complexities of Designing Voucher Studies 

Certain aspects of the research process in general, and of voucher programs in particular, can lead 
different analysts to conflicting conclusions even when they are using essentially the same data. 

Comparison groups of students and families. Determining which groups of students and 
families to study is one of the most complex parts of designing a voucher study and a source of 
considerable disagreement among researchers. Voucher studies usually try to draw out the effects 
of the voucher program by comparing students who use vouchers (or are offered them) with students 
who do not use vouchers but are otherwise similar in background and prior achievement. But people 
can vaiy in many subtle respects. There are several reasons why it is not easy to define these groups 
or design these comparisons. 

First, the broad group of families who do not use vouchers actually consists of several subgroups 
that may differ from each other — and from voucher users — in identifiable or unknown ways. 
The “non-users”include: (a) those who were never eligible; (b) those who were eligible but were 
not aware of their eligibility; (c) those who were aware of their eligibility but didn’t apply; (d) those 
who applied but were not selected by lottery or were rejected for other reasons; (e) those who were 
selected but could not use the voucher due to financial or similar factors; and (f) those who formerly 
used vouchers but left the program voluntarily or involuntarily. It is quite possible that families 
who actively choose vouchers may vary in key respects from those who do not; that successful 
applicants may vaiy from unsuccessful ones; that students who leave voucher programs may differ 
from those who stay; and so on. 

Policymakers and the public sometimes overlook a critical distinction between offering a voucher 
and using it. Studies of publicly funded vouchers have focused primarily on the users; however, 
closer examination of the “offered-but-didn’t-use” group could shed light on whether the act of 
offering a voucher has impacts on families, along with the reasons why some families don’t use 
vouchers. 

Second, many researchers feel strongly that the best way to isolate the effects of a voucher program 
from the multiple factors that influence education is to compare students who received vouchers 
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through a random selection process, such as a lottery, with those who did not. Other researchers, 
however, are skeptical about whether a true, random-assignment experiment is possible within the 
rules and conditions that govern student selection in current publicly funded voucher programs 
(although it may be more possible in some privately funded voucher programs). And some 
researchers contend that there are fundamental differences between the families who are selected 
for a voucher program and therefore receive the choice they desired, and the families who are not 
selected and must return to public schools they were very likely dissatisfied with in the first place. 

Third, researchers use statistical techniques to control for background differences between program 
users and an available comparison group. But researchers have different opinions about which 
background differences are meaningful and how to adjust for them. 

Simultaneous reforms. Voucher legislation has often been implemented at the same time as 
other education reforms in the same system; for example, public school choice in Milwaukee or 
the highly visible ratings of public schools in Florida. This makes it difficult to isolate the specific 
effects of vouchers. 

Incomplete or missing data. Voucher researchers are often confronted with incomplete or 
missing data, as a result of high mobility in low-income communities, attrition in voucher programs, 
difficulties of tracking the same individuals over several years, and reluctance of some individuals 
to answer survey questions. In some cases, students do not provide complete answers to tests, and 
in other cases, schools systematically exclude from testing certain students, such as students with 
disabilities or English language learners. Investigators must figure out ways to adjust for these 
gaps, but if there are too many it can compromise the whole study. 



What We Know about Impacts on Students 

Studies of current publicly funded voucher programs have focused primarily on the achievement 
of students who use vouchers. Other student outcomes, such as educational attainment, motivation, 
attendance, and post-school success, have received far less attention. 

Studies of publicly funded voucher programs have reached mixed conclusions about 
whether students who use vouchers improve their achievement compared with students 
who remain in public school . 

Overall findings. Some researchers have found no significant improvement in the achievement 
of students who use vouchers; others have found gains in one or two areas, such as mathematics or 
vocabulary; and others have found significant gains across more than one subject. 

Student achievement in Milwaukee. Three major studies — by the Witte team, the Greene 
team, and Rouse — reached varying conclusions about whether achievement improved for 
Milwaukee voucher students in private schools. All three used achievement data from the earlier 
years of the program, because no new data has been collected since 1995. But the studies employed 
different research methods and compared different groups of students. 

Rouse and the Greene group found statistically significant positive outcomes for students in 
mathematics, while the Witte team did not. The Greene group found significant positive outcomes 
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in reading, while Rouse and the Witte team did not. The Witte study also concluded that voucher 
students who left the program for various reasons had lower test scores than those who continued 
to participate. 

Student achievement in Cleveland. Two teams of evaluators — the Metcalf group and the 
Greene group — have studied achievement of Cleveland scholarship students. They looked at 
different subsets of pupils, used different methods, and reached varying conclusions. 

In 1997 and 1998, the Metcalf team analyzed the achievement of students who had attended 
Cleveland public schools in the second grade (and therefore had baseline test scores) but had 
attended private schools with vouchers in third or fourth grade. Their achievement was compared 
with that of a group of public school third graders, with certain controls for background character- 
istics. The study found no significant differences in achievement between the two groups at the 
end of the first year. By the end of the second year, the study found positive effects in language and 
science for voucher students on average. But according to this study, voucher students in new 
private schools performed significantly less well by the end of the second year than either the public 
school group or voucher students in pre-existing private schools. Students who left the voucher 
program after a year also performed less well than those who stayed. 

In 1998-99, the Metcalf team began to look at achievement and a range of other outcomes for several 
subgroups of students who entered the voucher program as first graders, and their families. The 
subgroups being examined include voucher users, unsuccessful applicants, students who were 
offered vouchers but turned them down, and those who didn’t apply. Findings about student outcomes 
from these later years are not yet available. 

The Greene group studied achievement of students attending two private schools newly established 
for voucher recipients. The investigators found significant gains for voucher students in reading 
and mathematics between fall and spring of 1996, and lesser but still significant gains in 1997. 
This same team also did its own analysis of the Metcalf third-grade data, and concluded that voucher 
students achieved significantly higher than public school students in language and science. 



What We Know about Families Who Use Vouchers 

Studies of Milwaukee and Cleveland have looked at which parents choose vouchers and why, and 
whether they are satisfied with their choice. 

The main reasons why parents say they use vouchers are to find a better quality education 
and safer learning environment for their children . 

Reasons for using vouchers. Milwaukee parents surveyed said educational quality was the 
most important factor in their decision to participate in the voucher program, followed by teaching 
approach and style, disciplinary environment, and general school atmosphere (Witte, 1999). 
Cleveland parents said the reasons they applied for vouchers were academic quality, safety, school 
location, and religion. Many Cleveland parents who did not use vouchers said they were not aware 
they had been offered one. Among those who actively turned down vouchers, the main reasons 
given were transportation problems, financial considerations, or failure to gain admission to the 
private school of their choice (Greene et al, 1997). 
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Parent information. Most participating families in Milwaukee learned about the voucher program 
from friends and relatives (Witte, 1999; Wisconsin Legislative Audit Bureau, 2000). The Wisconsin 
Legislative Audit Bureau recommended that voucher program administrators should do more to 
make families aware of their eligibility status, inform them about the program’s requirements and 
the fees involved, and help them make an informed choice. 

Milwaukee and Cleveland families who use vouchers are primarily African American, have 
lower incomes than typical public school families, and are more likely to be headed by 
a single parent Compared with low-income public school families, voucher families 
appear to be smaller, better educated, and somewhat more involved in their children's 
education . 

Characteristics of voucher^using families. In both cities, the racial and ethnic composition 
of families who used vouchers was quite similar to the demography of the public school systems 
(Wisconsin Legislative Audit Bureau, 2000; Metcalf, 1998). 

In Milwaukee, voucher-using families were more likely to be poorer, smaller, and headed by single 
parents than low-income public school families. However, the mothers were more likely to have 
attended some college, had higher educational aspirations for their children, and were more involved 
in their children’s education at school and at home (Witte, 1999; 1995). In Cleveland, the Metcalf 
evaluation found that while voucher applicants were similar in family structure and income to their 
public school counterparts, they had more years of education, on average, and were more interested 
and involved in their children’s education. 

Both the Witte study and the Metcalf study concluded that voucher programs can successfully be 
structured to target low-income families. 

Prior achievement of voucher students. Voucher users in Milwaukee entered the program 
with significantly lower average achievement in reading and math than their counterparts who stayed 
in public schools, according to the Witte study — a finding that suggests the program does not 
skim the most qualified students from the public schools, as some have feared. In Cleveland, 
however, students entering the voucher program had somewhat higher achievement than their public 
school peers, according to the Metcalf study. 

Parents who use vouchers in Milwaukee and Cleveland are quite satisfied with the 
education in their chosen private schools. 

Parent satisfaction. The Witte evaluation of Milwaukee found that parents who used vouchers 
were more dissatisfied with their child’s former public schools than low-income public school 
parents, but were more satisfied with their current schools. In Cleveland, both the Metcalf team 
and the Greene group found considerable satisfaction among voucher parents with their chosen 
schools. 
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What We Know about Vouchers and Schools 

The effects of vouchers on public, private and religious schools have not been closely studied. 
Current research does shed light on the characteristics of schools involved in voucher programs 
and the impact of vouchers on availability of private education. 

Even before the implementation of voucher programs, public and private schools differ 
on some key features . 

In the Cleveland voucher program, participating private schools had much smaller classes and 
lower enrollments than public schools. But public schools had teachers with more experience and 
more coursework beyond the bachelor’s level (Metcalf, 1998). These differences are consistent with 
national trends. 

Voucher programs can help to maintain a viable private school sector in urban areas . 

In Milwaukee, the voucher program has allowed some urban private schools to come back from 
the brink of insolvency, and helped others to expand their enrollments and facilities (Witte, 1999). 
Not all voucher schools survive, however; a few have gone bankrupt or closed. But in general, it 
appears that vouchers can increase the private school alternatives in a community. 



What We Know about Voucher Program implementation 

The Milwaukee and Cleveland voucher programs have undergone reviews of their management 
and implementation. A few broad findings have emerged. 

Audits in Milwaukee and Cleveland have uncovered some compliance and administrative 
problems in voucher programs and have recommended better monitoring and enforcement 

Voucher program reviews in Milwaukee. Independent reviews of the Milwaukee program 
have highlighted problems with transportation reimbursements, including excessive taxi expendi- 
tures for transporting voucher students, and have recommended improved administrative 
procedures (Wisconsin Legislative Audit Bureau, 1999; Witte, 1994). 

After investigating a complaint about student selection in several private schools, the Wisconsin 
Department of Public Instruction has issued findings of probable cause against seven schools for 
violating the voucher program requirements concerning student admissions and eligibility. A 
different review by the Wisconsin Legislative Audit Bureau (2000) recommended that state 
administrators improve monitoring of program requirements by conducting annual site visits to a 
sample of private and religious schools. 

Program reviews in Cleveland. A review in Cleveland uncovered problems with recordkeeping, 
data verification, transportation, and communication in the voucher program. Some schools did 
not receive voucher payments on time, and parents did not always get information they needed. 
Transportation, which is provided by the public school district for voucher students attending private 
schools, was sometimes unreliable, and parents were dissatisfied. The evaluators recommended 
several improvements (KPMG, 1999). Other audits also found that significant overpayments had 
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been made to transport voucher students to school (a finding that led to changes in the program’s 
transportation policies), and to provide vouchers to some 30 students whose family incomes far 
exceeded income guidelines (Deloitte & Touche LLP, 1997; Petro, January 1999). A second audit 
by the Ohio state auditor found that large overpayments had been made to one school during a 
three-year period for 70 students who were not actually enrolled, leading the auditor to recommend 
improved monitoring and verification of income and transportation invoices (Petro, September 1999). 

Cost issues. The Wisconsin Legislative Audit Bureau (2000) determined that although the 
Milwaukee public schools did not experience any significant change in total revenues as a result 
of the voucher program, this was because the district raised local property taxes to make up for the 
state aid it did not receive. Thus, the costs to Milwaukee property taxpayers was higher than it 
would have been in the absence of the voucher program,. The Bureau also noted that other public 
school districts in Wisconsin are required to contribute 0.6 percent of their general state aid to 
support the Milwaukee voucher program. In Cleveland, a management evaluation found no direct 
financial effects of the voucher program on Cleveland Public Schools in its early years (KPMG, 
1999). 
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WHAT WE DON’T KNOW 
BUT COULD KNOW 



Most key questions about vouchers have not been addressed or resolved by current studies, but 
that doesn’t mean they cannot be answered. Within the current structure of voucher programs, we 
could learn more about a host of issues, through additional high-quality research. 



What More Can We Learn about Milwaukee and Cleveland? 

There is still much to be learned about vouchers in Milwaukee and Cleveland. The fact that these 
programs have been in place for years complicates some research issues but helps with others. On 
one hand, the longevity of the programs makes it more difficult to isolate the specific effects of 
vouchers, because quality comparison data from the period before vouchers may no longer be 
available. On the other hand, the passage of time and the expanded scale of the programs enriches 
opportunities to study how programs evolve and what happens over time to students, families, and 
schools. 

Additional research in Milwaukee and Cleveland could build on earlier studies by digging more 
deeply into issues flagged by prior research and by expanding the inquiry into new areas. 

/Wore could be learned in Milwaukee and Cleveland about student participation, selection, 
attrition, achievement, and other student outcomes. 

Impacts on students. Additional studies in Milwaukee could tell us how the 1995 legislative 
changes, which raised the cap on program participation and brought in religious schools, have 
affected student participation. Are different types of students participating? Are students who use 
vouchers a representative sample of those eligible? 

Findings from state department investigations also suggest a need for more information about how 
private schools select and admit students. Questions also remain as to why some students leave 
the program (the attrition rate in the early years of the program ranged from 44% in the first year 
to 23% in the fourth year, according to Witte’s data). 

Although questions remain about student outcomes in Milwaukee, we are unlikely to learn more 
as long as schools are not required to collect data (an issue discussed later in this report). The 
complexities of determining an appropriate comparison group are also likely to remain. But if 
participation in the voucher program continues to rise toward its full potential, it may become 
possible to analyze the program’s impact on public school student outcomes. 
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In Cleveland, the Metcalf data comparing various subgroups of families and children is likely to 
yield more extensive information about the effects of voucher programs on achievement and other 
outcomes, such as motivation, involvement in school, attendance, and behavior. Studies by other 
groups could also illuminate various impacts on students. 

Further study in Cleveland could also determine how private schools respond when voucher students 
with behavior or academic problems seek admission and how public schools respond when students 
with such problems become eligible for vouchers. 

Additional studies could help clarify how and why parents choose schools, how they 
receive and use information, and how the voucher program affects families . 

Impacts on families. In Milwaukee, it would be possible to learn more about the kinds of 
information parents receive. What kind of information do they get about private schools and program 
requirements, and from which sources? By surveying the same groups of parents at different points 
in time, studies could also track the longer-term effects of the voucher program on participating 
and non-participating families. For example, how do vouchers affect parents’ involvement in their 
children’s education, family schedules and transportation, budgets and expenses, and other factors? 

In Cleveland, more research could clarify why only a small proportion of eligible parents actually 
applies for vouchers and how many explicitly choose the school their child attends. Additional 
parent surveys could shed more light on why parents make certain choices, whether they continue 
to be satisfied with their chosen schools, and how the uncertain legal situation has affected parents’ 
decisions. 

We need much more information about the impact of vouchers on curriculum, teaching, 
staffing, administration, and climate in private and public schools . 

Impacts on schools. Some of the least studied but most vital questions about vouchers pertain 
to their impacts on public schools. The expansion of the Milwaukee program makes it possible to 
investigate questions that could not be answered before. Research on school practices and 
environments can be relatively expensive, however, because it often requires surveys, classroom 
observations, interviews, or focus groups. To answer some questions, it may be necessary to compare 
data from schools before and after they became involved with the voucher program, then further 
compare them with a group of similar schools that do not accept vouchers. 

We could learn more about instructional differences between public and private schools and about 
student-teacher interactions with voucher and non-voucher students. Are particular instructional 
practices in private schools having promising effects for voucher students? If so, can they be 
replicated in other public or private schools? How do vouchers affect curriculum and instruction, 
staffing, discipline, and other practices in participating private schools? What do private schools 
do when voucher students have behavior or academic problems? If current voucher programs expand 
to full potential, it may become possible to study changes in public schools that lose pupils to voucher 
schools. 

Further research could also help answer questions about the supply of schools. It would be useful 
and feasible to track the conditions under which new private schools open to meet voucher demands, 




24 



as well as to study any expansion of existing private schools. It would also be possible to analyze 
the characteristics of newly established schools, such as tuition, mission, admissions policies, and 
hiring practices. 

Another recurring question is how voucher programs affect racial segregation and social stratification 
in schools. A study in Milwaukee (Fuller and Mitchell, 1999) concluded that private schools 
attended by voucher students are less racially isolated than the Milwaukee public schools. A study 
in Cleveland by Jay Greene (1999) concluded that private voucher schools, on average, are more 
racially integrated than the city’s public schools. An analysis by Mincberg and Holmes (1999) 
contended that the Fuller and Mitchell study had several analytical flaws and that its data was 
inadequate to support the study’s conclusion; it also criticized the Greene study for comparing the 
racial composition of voucher schools with that of the entire Cleveland metropolitan area, rather 
than with the city of Cleveland. Further analysis of issues of schools’ racial and ethnic makeup 
could be useful, especially analyses that tracked patterns over time. 

Additional studies could examine schools* compliance with program requirements and 
other administrative issues . 

Program implementation. The question of whether private schools are complying with 
procedures for selecting students merits further study. Other implementation issues that could be 
examined are whether actions of public schools have created implementation problems for private 
schools; how voucher schools in Milwaukee and Cleveland have reacted to charter school legislation; 
and which program requirements are the most and least effective in accomplishing the stated goals 
of a voucher program. 

Costs and financial issues could be studied in much more depth. 

Financial impacts. Much more could be learned in both Milwaukee and Cleveland about the 
financial impacts of vouchers on families, private schools, public schools, and the public education 
system. Data available in public records could help with these kinds of analyses, but it would have 
to be supplemented with information obtained from private and religious schools and parent surveys. 

Several questions warrant further study. How much does it cost families to use vouchers when fees 
and other non-reimbursed expenses are factored in? Do private and religious schools provide other 
subsidies to participating families, in addition to the vouchers they receive? What are the total 
costs of administering voucher programs and the total levels of expenditure per participant? How 
do vouchers affect the costs of education and tuition rates for private schools? How do per pupil 
expenditures for private school voucher students compare with those for public school students, if 
expenditures not related to voucher participants are excluded and if all expenses, including parish 
contributions and non-tuition resources, are included? If vouchers do not cover the typical costs 
of a private school’s tuition, does the school receive other kinds of subsidies? What are the cost 
differentials between private voucher schools and public schools, including special education and 
other special services? How do voucher programs affect the level of state and local funding available 
to the public school district? 
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More could be learned about the effects of the voucher programs on citizen support for 
public education . 

Public support for public education. How do voucher programs affect public attitudes about 
public education? How do they affect the public’s willingness to fund public schools? How do they 
affect public policy making? These are not easy questions to answer because there are no logical 
comparison districts with the same characteristics and contexts as Milwaukee and Cleveland. Even 
in these two cities, it may be difficult to find comparison data from before the voucher programs 
were instituted. Although studies of public support would have to acknowledge the difficulty of 
isolating the effects of vouchers from other reforms occurring at roughly the same time, it would 
still be possible to compare certain trends in cities with voucher programs and those without. Some 
useful sources of data might be voting behavior, surveys of public attitudes, records on changes in 
funding levels, and records from state and local governing bodies, such as school boards. 



What Can We Learn from Florida? 

The Florida evaluation team described earlier in this report intends to examine a wide range of 
questions. There are also opportunities for research by other groups, including smaller-scale studies 
that explore subsets of issues in depth. 

The size and statewide nature of the Florida voucher program could provide a unique 
opportunity to study compelling voucher issues on a systemic scale . 

If the courts ultimately allow the state to implement the voucher program, and if one-fifth of the 
students in the state eventually become eligible as some have projected, researchers could begin 
to learn more about voucher effects on public, private, and religious schools. The Florida program 
is also unique in its statewide scope and its policy of defining the target group for vouchers by 
public school ratings, rather than by family income. 

Because Florida administers statewide achievement tests and collects a range of other administrative 
data from public schools, researchers should be able to draw from a reasonably extensive data base. 
To learn more about aspects of private and religious schools not covered by state data, researchers 
could do surveys of private school teachers and administrators, perhaps in conjunction with case 
studies, observations, and other forms of qualitative research. Research on Florida will have to 
acknowledge the fact that the voucher program is closely intertwined with other reforms, including 
the grading of public schools. 

Florida may prove to be a valuable site for investigating the following kinds of questions: 

Effects on students. The state requires all public school students, as well as voucher students 
in private and religious schools, to take statewide achievement assessments, so researchers will 
be able to make achievement comparisons between voucher users and various subgroups of non- 
users. Do test scores or other outcomes improve for students who use vouchers? What happens to 
the performance of the students who stay in the public schools? Do voucher students participate 
in activities at their private schools that they would not have participated in if they had stayed in 
public schools? 
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Student selection. Which specific procedures are schools using to randomly select students to 
receive vouchers? Do some schools use disciplinary policies or other methods to discourage certain 
students from applying? Do private and religious schools admit or reject students with disabilities? 

Effects on families. What are the characteristics of students and families who use vouchers? 
Who is responsible for ensuring that parents receive school profile information? Do parents receive 
information in a form they can understand and readily use to compare schools? What other expenses 
will families with vouchers have to pay? What other expenses will they voluntarily choose to pay? 

Effects on schools. What are the effects of vouchers on participating private schools and on 
public schools? How do voucher programs affect the educational climate of public and participating 
private schools? To what extent does government regulation of these schools increase? How do 
private schools react to voucher requirements and goals? How does the voucher program affect 
classroom practices and teacher attitudes in public and private schools? How does it affect 
innovation in education? How do public schools respond to the voucher program; for instance, do 
they change how they use resources, recruit teachers, or hold people in the school accountable? 
Are there benefits to public schools due to increased competition from the private sector? 

Changes in school demographics. Does the voucher program affect the racial, ethnic, 
socioeconomic, or achievement composition of individual public or private schools? 

Supply and demand. If the Florida program grows, it may be possible to study its effects on the 
supply of private schools over time. Do new private schools form? Do existing private schools expand 
in response to rising demand? How does the program affect the price of private schooling? 

Program implementation. How are program requirements being interpreted in cases where 
the statute is vague? How are requirements being enforced, and who is generally responsible for 
enforcing them? Who enforces the private school requirements, such as those affecting teacher 
qualifications and disciplinary policies? How do parents lodge a formal complaint against a private 
school they believe is not complying? How effectively do public officials monitor compliance? 

Costs. How much does it cost to operate and administer the voucher program? What effect do 
vouchers have on public school administrative, transportation, and other costs? How does the 
voucher program affect the total financial resources available to public schools, once adjustments 
are made for losses in government aid and extra costs? 

Support for public education. What are the effects of the voucher program on citizen and 
political support for public education? How does the program affect citizen support for such goals 
as raising achievement of low-achieving students who remain in public schools? Does it influence 
people’s willingness to pay taxes for public education? What other changes occur in public 
involvement with public schools, such as time spent volunteering, funds raised by PTAs, or levels 
of business or community partnerships with schools? How does the voucher program affect voting 
patterns? 
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What Can We Learn from the International Context? 

To learn more from the international context does not necessarily require new studies. Several studies 
have already been done, but only a few efforts have been made to discern patterns across nations 
or bring their findings to a wider audience. Because existing international studies vary greatly in 
quality, scope, and methodology, it is difficult for Americans to determine which are the most 
credible. The international information that does make its way into U.S. voucher debates often 
comes from secondaiy sources, rather than from the original studies, so it is hard for people to 
know what has been left out or reinterpreted. 

Aiming the research on choice, voucher, and private school subsidy programs in other 
countries would be a relatively low-cost way to learn more about issues often raised in 
the US . 

What would be most helpful at this point is a comprehensive, balanced, and rigorous review that 
looks across existing studies of choice, voucher, and private school subsidy programs in other 
countries and that pulls the work together. 

This review should examine the impacts of these programs on students, families, schools, and 
educational systems. Tt should seek to reach hypotheses about issues for which we have few or no 
answers in the U.S., such as the effects of competition among different kinds of schools and the 
social effects of large-scale choice programs. Many of the international programs have been in place 
for several years and could offer insights into the long-term effects of government involvement with 
private and religious schools. Although these international programs operate in different educational 
and cultural contexts from ours, these very differences could shed light on how context and rules 
shape program operations and outcomes. 

The proposed review should examine both primary studies, in which researchers collect new data 
or conduct an original analysis, and secondaiy studies, in which researchers synthesize, critically 
analyze, or assess trends across studies done by others. The review should explicitly assess each 
study’s research methods; acknowledge its strengths, weaknesses, and limitations; and note whether 
its methodology is rigorous enough to produce confidence in its findings. Attention should also be 
given to the broader context in which the program operates, the extent to which findings can or 
cannot be generalized to the U.S. context, and the conclusions that can be drawn about which 
programs work best in a particular context. 

The work should be done by several individuals with expertise in the particular countries and 
with expertise in the issues and methodologies to be addressed. The research process should be 
structured to include input from people with diverse perspectives. Clear guidelines will need to be 
developed up front about which voucher and choice programs should be included in or excluded 
from the review. 
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WHAT WE CAN’T KNOW BECAUSE OF 
PROGRAM STRUCTURE 



Key questions about vouchers cannot be answered at present because of the structure or context of 
current voucher programs. Some of these questions could be answered if the legislation or regulations 
were changed. 



Why Some Questions Cannot Be Answered Now 

The elimination of evaluation requirements has essentially halted new research in 
Milwaukee . 

Lack of evaluation in Milwaukee. The elimination of evaluation requirements in the law has 
greatly hindered the study of numerous issues. A major question — what has happened with student 
achievement since 1995 — cannot be answered because the statute no longer requires schools to 
provide data for research, and there are no uniform testing requirements. To learn more about 
Milwaukee will require either changes in the law, or unprecedented funding and powers of 
persuasion on the part of researchers. Without a legislative mandate, evaluators would have to 
persuade public, private, and religious school officials to voluntarily provide data for study. Or, 
evaluators could administer their own tests to voucher students and cany out other independent 
data collection and analysis. This latter option is costly and would not be very effective if it did not 
involve a representative cross-section of students. Strong participation of schools is also needed to 
produce other critical information about program implementation and systemic effects. 

Some questions cannot be answered in the current context of publicly funded voucher 
programs , but may be answered if current programs reach their full potential . 

Issues of marketplace competition. Under the current structure, rules, and voucher amounts 
of existing programs, and given the limited percentages of students affected, it may not be possible 
to investigate such questions as whether vouchers affect the educational “market” in the ways 
proponents or opponents predict. But if programs expand as projected, researchers may be able to 
learn more about such issues as whether increased competition from private schools spurs 
improvements in public schools or causes inferior schools to close and how rising demand for 
vouchers influences the supply and cost of private schools. Studies might also illuminate whether 
competition from vouchers encourages the higher-achieving students to leave public schools, affects 
public school teacher motivation, or produces other negative effects that opponents have predicted 
in public schools. 
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Issues of context. It may not be possible to know whether the results in Milwaukee or Cleveland 
could be replicated in different contexts. Moreover, because Milwaukee, Cleveland, and Florida 
have undertaken other types of school reforms at the same time as vouchers, it would be difficult 
to sort out the systemic effects attributable to vouchers. Researchers have developed various methods 
to isolate these effects, but in some situations, the causes will remain murky. 

Collecting data from schools creates challenges for research . 

School data. Gathering information about any type of school — public, private, or religious — 
is a challenge. Researchers who study vouchers depend on the cooperation of school officials to 
gain access to vital data. Private school data collection, which is central to the study of vouchers, 
raises special challenges. Although state education agencies generally collect various kinds of data 
about public schools, they do not collect much from private and religious schools, which are outside 
the sphere of state administration and regulation. The Cleveland and Florida programs do require 
some data collection from participating private schools, but not very much. And in any case, 
evaluators still face the problem of how to assess the effects of the voucher program on non-voucher 
students in participating private schools. 

Private and religious schools (as well as public) may have little incentive to open their schools to 
evaluators or to add to their paperwork burdens and costs. In addition, private schools vary 
considerably in their affiliations, missions, and characteristics. Even if they are willing to share 
their data, such as student test scores, the data may be too incompatible and inconsistent for research 
purposes. 



General Observations 

Three more points should be made about what we can’t learn. 

First, it is not realistic to expect studies to give a clear answer to every question about vouchers. 
Some issues are likely to remain unresolved even with additional studies, because of the design 
challenges already described and the inherent limitations of the research process. 

Second, every study has budget, time, and staff limitations. Certain information that could be useful 
is often complicated or costly to obtain. Some questions could theoretically be answered, but only 
through very expensive or impractical methods. Researchers and policymakers must constantly 
weigh issues of costs, practicality, and usefulness. 

Third, people support or oppose vouchers for many different reasons. There is no magic question 
or answer that will si nglehandedly resolve the policy debate. Even with fuller information, people 
will still have to make tradeoffs between competing or complementary policy goals. No matter how 
good the research is, political, legal, and philosophical considerations will continue to shape 
decisions about vouchers. 




30 



CONCLUSION 



Because of recent studies of publicly funded voucher programs, we know more about vouchers 
than we did a decade ago, when policy debates in Wisconsin and elsewhere dealt mostly in the 
realm of the hypothetical. We now have a baseline of evidence on some pressing questions about 
vouchers. But we could learn much more, if policymakers, researchers, and funders of research 
were to take certain steps. 

The Center has recommended some actions that could be taken. They appear at the beginning of 
this report. Some of these actions could be done right away, such as incorporating certain questions 
into ongoing studies of vouchers. Others, such as an international review, could be initiated as 
soon as funding is found. Some actions will take considerably more time and effort, such as starting 
up new research or changing legislation and regulations. 

The agenda laid out in this report is ambitious and will require funding, commitment, cooperation, 
and good will to make it happen. But the payoff will be improved public understanding and better 
evidence on which to make policy decisions. And that will help both proponents and opponents of 
vouchers and the whole country. 



For more information , visit the Center’s Web site at www.ctredpol.org. Posted on the site are the text 
of the voucher subcommittee reports ; a list of voucher subgroup members and other participants in 
the voucher meetings convened by the Center ; and the Center’s other publications. 
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X 



Major Research Questions That Should Be Addressed 
by Evaluations of Any Voucher Program 

1. Questions about the nature of the voucher offer. 

O How much is the subsidy? What portion of private school costs is it likely to cover? 

O Where can the voucher can used? 

O Who determines which private schools may accept voucher students? How is this 
determined? Who enforces these rules? 

O What must students and parents do to use the voucher? 

O Which ancillary services are provided, such as transportation or information to parents 
about schools? 

Q Are information, guidance, counseling, transitional assistance, financial assistance, and 
other supports provided to help students and families use the voucher and access and thrive 
in their new schools? 

O What happens to public funds if a student is asked to leave a private school, is expelled, 
or transfers to another school? 

2. Questions about requirements on schools where vouchers may be used. 

O What regulations accompany the voucher? Who enforces them? 

0 Can a private school’s enrollment be made up of a majority of voucher students? 

O Do private schools adhere to local education standards? If so, do elected school boards 
have any power over the private school’s curriculum? 

O Does the fiscal stability and longevity of a private school determine whether it may receive 
vouchers? If so, who determines this? 

O How do private schools react to these requirements and goals? 

0 Do private school students take the same assessments as public school students? Is test 
information available for public scrutiny? 
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3* Questions about constitutional considerations, 

O Does the potential use of vouchers in private schools violate state or federal constitutions? 

O Are there constitutional challenges pending against the voucher program? 

O Can children “opt out” of participating in a private school’s religious activities at the request 
of their parents? 

O Can voucher funds be used to pay for religious activities? If not, how do private schools 
assure that public funds are not used for religious purposes? 

4* Questions about the selection of students, 

O Which students are eligible for the voucher under the program’s rules? Who enforces these 
rules? 

0 How are students selected to receive the voucher? Who ensures that the selection is carried 
out as prescribed? 

O Can a private school reject a student based on a physical or mental disability? Are students 
with disabilities discouraged from applying? Are special services available in the private 
school for students with disabilities? 

0 Are voucher students required to take private school admissions tests? Can students be 
rejected based on their scores? 

O Can a private school reject voucher students based on their academic record, lack of English 
proficiency, discipline record, or other factors? 

0 Do schools give preferences in admissions (for example, to parishioners, siblings, or 
financial contributors)? 

O What are the characteristics of the students in the eligible population? 

0 How do they compare with other students in the district or school? 

5* Questions about the use of vouchers, 

0 Who uses the voucher and who does not? Does use vary by race, gender, class, parents’ 
education, students’ prior achievement, or other factors? 

O How long do students continue to use vouchers? 

O Why do some voucher students leave private schools? 

O Why do people use and not use vouchers? 

0 At which schools do people use vouchers? How do they decide on a school? 

0 What educational services do students receive in their chosen schools? How do these 
services compare with those provided by public schools? 

0 Do private schools require parent involvement for voucher students and non-voucher 
students? 
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Questions about the impacts of the voucher* 

Impacts on students who use vouchers or are offered vouchers: 

O What are the impacts of vouchers on students’ attendance, discipline, achievement, 
educational attainment, attitudes toward school, and beliefs about acceptance of diversity 
and teaching a common set of values? 

O If an evaluation finds changes in student achievement, what are the likely reasons 
underlying these changes? For example, are improvements in the achievement of voucher 
students attributable to the nature of educational experience, such as smaller class sizes 
in private schools? 

Impacts on the parents and families of voucher students: 

O What are the impacts of vouchers on parents’ satisfaction with the school’s educational 
services? 

O Do the values the school promotes conform with the values held by the parents of voucher 
students? 

O How does using a voucher affect the family’s daily schedule, especially if transportation 
issues are involved? 

O How do vouchers affect the level of parents’ involvement with and support for schools? 

Impacts on the education of students who do not use or are not eligible for 

vouchers: 

O What are the impacts of the voucher program on the resources available for other students? 

O How do vouchers affect innovation in education? 

O Does the voucher program change the student body in public schools? 

Impacts on eligible private schools: 

O Do private schools in the community have the capacity to handle new voucher students? 

O Do new private schools emerge? 

O If private schools hire new teachers, what are their qualifications and background compared 
with other private and public school teachers? 

Q Do existing private schools change their educational services? 

O What is the impact of the voucher program on the composition of the student body? 

O How do vouchers affect the financial resources available in private schools? 

O How do vouchers affect private school policies in such areas as entrance requirements, 
discipline, and student tracking? 
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Impacts on public schools: 

O How does the voucher program affect the financial resources available to public schools, 
including any loss of government aid, additional costs for transportation and administrative 
oversight, and other resources? 

O What is the impact of vouchers on educational services in public schools? 

O What is the impact on the demographic mix of students in the public school system? 

O In the case of Florida or similar programs, how does a judgment that a school is failing 
affect the attitudes and actions of teachers, students, and parents toward their school? 



7 * Questions about cost implications. 

O What are the costs of private school services under a voucher program? 

O What are the costs of public school services? 

0 What are the costs of vouchers for families and students, including such non-monetary 
costs as time? What are the costs of transportation? What are the costs of private school 
services not covered by vouchers, and who pays for them? Are families informed of these 
additional costs before they choose a school? 

O What are the administrative costs of the voucher program? For example, what are the costs 
of providing parents with information about the program and schools, overseeing the 
selection process, and monitoring compliance with program requirements? 

8. Questions about effects on public support for public education. 

0 What are the effects of the voucher program on support for public financing of education? 

O What are the effects on support for public schools? 

O What are the effects of support for public educational goals, such as raising the achievement 
of low-achieving students? 
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